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(54) Pattern recognition apparatus for detecting predetermined pattern contained in input signal 



(57) A pattern recognition apparatus for detecting a 
predetermined pattern contained in an input signal is 
provided with plural detecting processing parts and 4A 
for detecting respectively different features for a same 
input, plural integrating processing parts for spatially in- 
tegrating, for each process results, the features detect- 
ed by the plural detecting processing parts, plural de- 
tecting memories for retaining the process results of the 



detecting processing parts, plural integrating memories 
for retaining the process results of the integrating 
processing parts, a global data line 1 030 to which all the 
predetermined detecting processing parts and all the 
predetermined integrating memories are connected at 
a certain timing, and plural local data lines each of which 
is connected to a predetermined set of the detecting 
processing parts, the integrating processing parts and 
the detecting memory. 



< 

CO 

o 

O) 
CM 

to 

CM 

T— 

£L 
LU 



FIG. 1A 



1010 _ 



1000- 



C0MTR0L 
PART 



INTEGRATING 

MEMORY 

C0NTR0UER 



INTEGRATING 
ADDRESS LINE 
1020 



1111C 

GLOBAL 
DATA SIGNAL 
1030 



1111B 



1021 
INTEGRATING 
MEMORY 
CONTROL 
SIGNAL 



INTEGRATING 
~H MEMORY 



1022 



1023 



1011 



WTEGRATNG 
MEMORY 



M012 



INTEGRATING 
MEMORY 



1013 



1090 



DETECTING PROCESSING 
PART CONTROL SIGNAL 

3 11A torn 

^^DETECTNG 
jr ADDRESS LINE 
1080 



DETECTING 
MEMORY 
CONTROLLER 




1042A- 



1042B 



1052A- 



DETECT WG 
PROCESSING PART 




LOCAL 
DATA UNE 

|45° 1072 
d 



DETECTING 
MEMORY 



EXTERNAL 
I/F 



1082 



-1162 



TO FKi IB- 



Printed by Jouve, 75001 PARIS (FR) 



(Cont. next page) 



EP 1 262 908 A1 



FIG. 1B 



1024 



1025 
AUXILIARY 
MEMORY 
CONTROL 
SIGNAL 



1014 



NTEGRAUNG 
MEMORY 



AUXILIARY 
MEMORY 

~r~ 

1015 



EXTERNAL 
I/F 

1130 



FROM FIG. 1A 

y\ 



1043C 



10438 



1 



DETECTING 
PROCESSING PART 



1043A 
1053A-^d INTEGRATING 



PROCESSING PART|*i 
I 



1053B 



1053C 












1044A-xi 
1054A^p 







1044C 



1 



044B 



DETECTING 
PROCESSING PART 



INTEGRATING 
PROCESSING PART 



T 054BT 



1054C 



L0CAL 

DATA LINE 1163 
1064 10 74 



DETECTING 
MEMORY 



EXTERNAL 
I/F 



T 
1164 



1084 



2 



1 



EP 1 262 908 A1 



2 



Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to a circuit con- 
figuration of a pattern recognition apparatus for execut- 
ing pattern recognition, detection of a specified object 
etc. by parallel operation of a neural network or the like. 

Description of Related Art 

[0002] Conventionally, the process of image recogni- 
tion or voice recognition is divided into a type of execut- 
ing In succession a recognition process algorithm spec- 
ified for a certain object of recognition as a computer 
software, and a type of executing such algorithm by an 
exclusive parallel image processor (SIMD, MIMD ma- 
chine etc.). 

[0003] As for the latter, the Japanese Patent Applica- 
tion Laid-Open No. 6-6793 discloses an object recogni- 
tion apparatus in which plural image processor units are 
employed to execute the process by DSP's provided in 
such processor units, and the obtained plurat results are 
transferred to another unit for executing recognition of 
the object. For example an image is divided into plural 
areas which are processed in parallel respectively by 
the processor units, and the recognition of object is es- 
timated in another processor unit by a neural network 
or fuzzy control. 

[0004] Also for a hardware for executing hierarchical 
parallel processing by neural network, the Japanese 
Patent No. 2679730 discloses the architecture of a hi- 
erarchical neural network enabling to realize a multi-lay- 
ered architecture by time-shared multiplex use of a sin- 
gle-layered hardware. It intends to realize multiple lay- 
ers in equivalent manner by time-shared multiplex use 
of a single-layered hardware, and provides a neural net- 
work, formed by mutual connection of plural neuron 
models and comprising a single-layered unit assembly 
including plural neuron model units each of which is ca- 
pable of generating a product of a time-shared multiplex 
analog signal with an external digital weighting data, in- 
tegrating such product by time-shared addition through 
a capacitor and outputting a voltage through a non-lin- 
ear output function in time-shared manner, a feedback 
unit for feeding the output of the aforementioned single- 
layered unit assembly back to the input unit thereof, and 
a control unit for executing control for time-shared mul- 
tiplexing of the analog signals from the units constituting 
the aforementioned single-layered unit assembly and 
for time-shared multiplex use of the aforementioned sin- 
gle-layered unit assembly through the feedback unit, 
whereby the neural network of a hierarchical structure 
in equivalent manner is constructed by the time-shared 
multiplex use of the single-layered unit assembly. 
[0005] Also the U.S. Patent No. 5,892,962 discloses 



a processor as a hardware employing FPGA (field pro- 
gramming gate array). In such processor, each FPGA 
is provided with a memory for holding the result of 
processing in FPGA, and the processing is executed by 
5 reading the results in such memories. 

[0006] Among such conventional technologies, the 
object recognition apparatus disclosed in the Japanese 
Patent Application Laid-Open No. 6-6793 is capable of 
processing in several stages for the areas assigned to 
10 the image processor units and for the further divided 
smaller areas back to the original areas, but Is incapable 
of hierarchical processing by parallel process with other 
plural processor units for the obtained plural process re- 
sults. It is also not capable of reading the results of 
15 processing, nor capable of spatially integrating the re- 
sults from the areas. 

[0007] Also the hierarchical neural network disclosed 
in the Japanese Patent No. 2679730 is associated with 
a drawback that the types of the practically realizable 
processes are quite limited because of the absence of 
a unit for variably and arbitrarily controlling the Interlayer 
coupling. 

[0008] Also the FPGA-based processor disclosed in 
the U.S. Patent No. 5,892,962 requires complex wirings 
in order to read out the intermediate results retained in 
the memories. 

SUMMARY OF THE INVENTION 

[0009] In consideration of the foregoing, an embodi- 
ment of the present invention provides recognition ap- 
paratus capable, with a single circuit configuration not 
requiring complex wirings, of executing hierarchical 
processing such as parallel processing with plural proc- 
ess units on the plural results obtained by processing in 
plural processors, also of spatially integrating the results 
of processing in the processors, and easily reading the 
result of each process. 

[0010] According to one aspect, the present invention 
relates to a pattern recognition apparatus for detecting 
a predetermined pattern contained in an input signal, 
comprising plural detection process means for respec- 
tively detecting a feature for a same input, plural inte- 
grating process means for spatially integrating the fea- 
tures detected by the plural detection process means 
for each process result, plural detection memories for 
retaining the process results of the detection process 
means, plural integrating memories for retaining the 
process results of the integrating process means, a 
common data line to which all the predetermined detec- 
tion process means and all the predetermined integrat- 
ing memories are connected at a certain timing, and plu- 
ral local data lines each of which is connected to a pre- 
determined set of the detection process means, the in- 
tegrating process means and the detection memory, 
wherein in entering the process results of the detection 
process means retained in the detection memories into 
the integrating process means, the data of a same ad- 
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dress are read from the plural detection memories and 
entered into the integrating process means, and, In en- 
tering the process results of the Integrating processing 
means retained in the integrating memories into the de- 
tection process means, the same data are read from the 
Integrating memories and entered into the plural detec- 
tion process means. 

[001 1 ] Other objectives and advantages shall be ap- 
parent to those skilled in the art from the description of 
embodiments of the invention which follow. In the de- 
scription, reference is made to accompanying drawings, 
which form a part thereof, and which illustrate embodi- 
ments of the invention. Such embodiments, however, 
are not exhaustive of the various embodiments of the 
invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0012] 

Fig. 1 is comprised of Figs. 1 A and t B showing the 

configuration of a first embodiment; 

Fig. 2 is a view showing a convolutional network 

structure; 

Fig. 3 is a flow chart showing the function of the first 
embodiment; 

Fig. 4 is a view showing feature amounts; 
Fig. 5 is a view showing an example of feature de- 
tection; 

Fig. 6 is comprised of Figs. 6A and 6B showing the 

configuration of a second embodiment; 

Fig. 7A is a view showingthe concept of the function 

of the second embodiment, in a state where a global 

line data switch is turned on; 

Fig. 7B is a view showingthe concept of the function 

of the second embodiment, in a state where a global 

line data switch is turned off; 

Fig. 8 is comprised of Figs. 8 A and 8B showing the 

configuration of a third embodiment; 

Fig. 9 is comprised of Figs. 9A and 9B showing the 

configuration of a fourth embodiment; 

Fig. 10 is a flow chart showing the function of the 

fourth embodiment; 

Fig. 11 is a view showing the timing of reconstruc- 
tion and processing In the fourth embodiment; and 
Fig. 12 is a view showing the configuration of an 
example in which the pattern recognition apparatus 
embodying the present invention is utilized in an im- 
age pickup apparatus. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[001 3J In the following, the present invention will be 
clarified in detail by preferred embodiments thereof, with 
reference to the accompanying drawings. 



(First embodiment) 

[0014] Figs. 1 A and 1 B are views showing the config- 
uration of the present embodiment. 

5 [0015] Referring to Figs. 1A and IB, a control part 
1 000 Is a unit for controlling the entire circuit, communi- 
cating with an integrating memory controller 1010 to be 
explained later, detecting process (processing) parts 
1 041 A to 1 044C, integrating process (processing) parts 

10 1051 A to 1054C and a detection memory controller 
1 090 and controlling these parts based on a recognition 
algorithm to be used thereby executing recognition. 
[0016] The integrating memory controller 1010 con- 
trols integrating memories 1 01 1 to 1 01 4 and an auxiliary 

15 memory 1015 thereby outputting data from these mem- 
ories to a global data line 1030 and writing the data on 
the global data line 1030 into the integrating memories 
1011 to 1014 or the auxiliary memory 1015. More spe- 
cifically, it executes the aforementioned operations by 

20 outputting an address to ah integrating address line 

1 020 and controlling integrating memory control signals 

1021 to 1024 and an auxiliary memory control signal 
1 025 such as a chip select signal for selecting the mem- 
ory to be operated and a write enable signal for selecting 

25 signal writing and signal readout. 

[0017] Generation of plural addresses can be 
achieved in the control part 1 000 by setting start and 
last addresses and a step number in the integrating 
memory controller 1 010, and by counting up a counter, 

30 storing the aforementioned start address, to the last ad- 
dress by the aforementioned step number. Various 
changes in such address generation allow to achieve 
data readout and reading in an arbitrary area in the in- 
tegrating memories 1011 to 1014 and in the auxiliary 

35 memory 1 015, thereby avoiding cumbersome wirings. 
[0018] The integrating memories 1011 to 1014 are 
used for retaining the process results of integrating proc- 
ess parts 1051 A to 1054C to be explained later. These 
memories are respectively connected to the global data 

40 line 1 030 and are adapted to output the retained results 
to the global data line 1030 or to fetch the process re- 
sults of the integrating process parts 1051 A to 1054C 
from the global data line 1 030 according to the integrat- 
ing memory control signals 1021 to 1024. 

45 [001 9] The auxiliary memory 1 01 5 is used for tempo- 
rarily storing the signal to be recognized, such as an im- 
age. Also this memory is connected to the integrating 
address line 1020 and the global data line 1030, and 
outputs the retained signal to the global data line 1030 

50 or fetches the data on the global data line 1 030 based 
on the auxiliary memory control signal 1025 from the in- 
tegrating memory controller 1010. 
[0020] On an integrating address line 1020, the inte- 
grating memory controller 1 01 0 outputs a signal indicat- 
es |ng the address of the integrating memories 1011 to 
1014 and the auxiliary memory 1 01 5. By changing such 
address, it is rendered possible to avoid the issue relat- 
ing to the wirings and to easily read the various process 
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results retained in the Integrating memories 1011 to 
1014, whereby the detecting process parts 1041A to 
1044C can execute process for signals obtained by 
combining these results. 

[0021] The integrating memory control signals 1021 
to 1 024 are used for selecting the integrating memories 
1 01 1 to 1 01 4 and distinguishing or controlling the write* 
in and read-out thereof. By executing the memory se- 
lection on time-shared basis, it is rendered possible, at 
the data read-out, to output the data of the integrating 
memories 1011 to 1014 to the global data line 1030 on 
time-shared basis, and, at the data writing operation, the 
memories are switched in synchronization with the inte- 
grating process part control signals 1121 A to 1124C to 
retain the process results of the integrating process 
parts, outputted on the global data line .1030, in the in- 
tegrating memories 1011 to 1014. 
[0022] The integrating memory control signal 1 025 is 
used for selecting the auxiliary memory 1015 and dis- 
tinguishing or controlling the write-in and read-out there- 
of. 

[0023] The global data line 1030 is connected to the 
integrating memories 1011 to 1014, the detection proc- 
ess parts 1041 A to 1044C, the integrating process parts 
1051 A to 1054C and the auxiliary memory 1015. There- 
fore, the data of the integrating memories 1011 to 1 014 
are entered into the detecting process parts 1041 A to 
1 044C at a same timing in parallel manner, and, by the 
time-shared process, the process results from the inte- 
grating process parts 1051 A to 1054C are written into 
the integrating memories 1011 to 1014 on time-shared 
basis. 

[0024] The detecting process parts 1041A to 1044C 
respectively share the processes required for the rec- 
ognition (for example edge detection). Therefore, the 
apparatus of the present invention executes the recog- 
nition process In total, by combining the processes of 
the detecting process parts. 

[0025] The detecting process parts 1041 A to 1044C 
execute process on the data entered through the global 
data line 1 030, and output the results respectively to the 
local data lines 1 061 to 1 064. The process results of the 
detecting process parts 1041A, 1041B, 1041C are out- 
putted to the local data line 1061, and those of the de- 
tecting process parts 1042Ato 1042C are outputted to 
the local data line 1062. The process results of other 
detecting process parts are also similarly outputted. 
[0026] The detecting process parts 1041 A to 1044C 
receive a same signal through the global data line 1 030, 
but execute different processes on such input signal. 
The result of such process, for example of the detecting 
process part 1041 A, is outputted to the local data line 
1016 and retained in the detection memory 1071 . Sim- 
ilarly, the process results of the detecting process parts 
are retained in the different detection memories 1071 to 
1074. Detecting process part control signals 1111 A to 
1114C indicate the detecting process parts to be oper- 
ated. The detecting process part control signals 1111 A 



Is connected to the detecting process part 1 041 A, 1 1 1 1 B 
is connected to the detecting process part 1041 B and 
so on (in Figs. 1 A and 1 B, there are only shown the de- 
tecting process part control signals 1 1 1 1 A to 1 1 1 1 C). For 

5 example, at a certain timing, the detecting process part 
control signals 1 1 1 1 A, 1 1 1 2A, 1 1 1 3A, 1 1 1 4A are enabled 
to operate the detecting process parts 1041 A, 1042A, 
1043A, 1044A, and the process results thereof are re- 
tained In the detection memories 1071 to 1074 respec- 

10 tively through the local data lines 1 061 to 1 064. Also at 
another timing, other detecting process part control sig- 
nals are enabled. 

[0027] The integrating process parts 1 05 1 A to 1 054C 
execute integrating processes on the data entered 
f5 through the local data lines 1061 to 1064, and output 
the results of such processes on the global data line 
1030. The signals to the integrating process parts 
1 051 A to 1 051 C, 1 052A to 1 052C, 1 053A to 1053C and 
1 054A to 1 054C are entered respectively from the local 
20 data lines 1061, 1062, 1063 and 1064. Also within the 
integrating process part control signals 1121 A to 1124C 
(not shown in Figs. 1A and 1B) from the control part 
1 000, process part enable signals are used to select the 
Integrating process parts 1 051 A to 1 054C to be operat- 
es ed. Also within the integrating process part control sig- 
nals 11 21 A to 1124C, output signals are used for con- 
trolling the output timing of the process results of the 
integrating process parts 1051 A to 1054C to the global 
data line 1030. 

so [0028] More specifically, at a certain timing, the proc- 
ess part select signal of the integrating process part con- 
trol signals 11 21 A, 1122A, 1123A and 1124A are ena- 
bled to operate the integrating process parts 1051 A, 
1052A, 1053A and 1054A. Then, based on the output 
35 signals, the process results are outputted to the global 
data line 1030 in succession for example from the inte- 
grating process part 1051 A. In this operation, the inte- 
grating memory controller 1 01 0 is controlled in synchro- 
nization to retain the data on the global data tine 1030 
40 in succession in the integrating memory 1011. 

[0029] The local data line 1061 is connected to the 
detection memory 1071, the detecting process parts 
1041 A to 1041C and the integrating process parts 
1051 A to 1051C. Also the local data line 1062 is con- 
45 nected to the detection memory 1072, the detecting 
process parts 1 042A to 1 042C and the integrating proc- 
ess parts 1 052A to 1 052C, and the local data lines 1 063, 
1064 are also similarly connected. Therefore, the proc- 
ess results from the detecting process parts 1041 A to 
50 1 041 C, 1 042A to 1 042C, 1 043A to 1 043C and 1 044A 
to 1044C and respectively retained in the detection 
memories 1071, 1 072, 1 073 and 1 074. Also the data of 
each detection memory are entered into the different in- 
tegrating process parts, such as those from the detec- 
ts tion memory 1071 to the integrating process parts 
1051 A to 1051C and those from the detection memory 
1 072 to the integrating process parts 1 052A to 1 052C, 
at a same timing in parallel manner. 
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[0030] The detection memory 1071 retains the proc- 
ess results from the detecting process parts 1041 A to 
1041C, while the detection memory 1072 retains the 
process results from the detecting process parts 1 042A 
to 1042C, and the detection memories 1073 and 1074 $ 
execute similar functions. The detection memories 
1 071 , 1 072, 1 073 and 1 074 are respectively connected 
to the local data lines 1061, 1062, 1063 and 1064, and 
output the retained data to the local data lines 1061 to 
1 064 or fetch the process results of the detecting proc- 
ess parts 1 041 A to 1 044C present on the local data lines 
1 061 to 1064, based on detection memory control sig- 
nals 1081 to 1084. 

[0031] On a detection address line 1 080, a detection 
memory controller 1 090 outputs a signal Indicating the 
address of the detection memories 1071 to 1074. By 
changing such address, it is rendered possible to avoid 
the issue relating to the wirings and to easily read the 
process results of the arbitrary position retained in the 
detection memories 1 071 to 1 074, whereby the integrat- 
ing process parts 1 051 A to 1 054C can execute process 
for the result of each area. 

[0032] The detection memory control signals 1 081 to 
1 084 are used for selecting the detection memories 
1 071 to 1074 and distinguishing or controlling the write- 
in and read-out thereof. 

[0033] The detection memory controller 1 090 controls 
the detection memories 1071 to 1074, thereby output- 
ting the data from these memories to the local data lines 
1 061 to 1064 or writing the data on the local data lines 
1061 to 1064 into the memories. More specifically, the 
aforementioned operations are executed by outputting 
an address to the detection address line 1081 and con- 
trolling the integrating memory control signals 1081 to 
1 084 such as a chip select signal for selecting the mem- 
ory to be operated and a write enable signal for distin- 
guishing the writing and reading operations. 
[0034] The detecting process part control signals 
1111 A to 1114C are used for communication between 
the detecting process parts 1041 A to 1044C and the 
control part 1 000, and are composed of a process part 
select signal for selecting the process part to be oper- 
ated among the detecting process parts 1041 A to 
1 044C, an output signal for permitting the output of the 
process results to the local data lines 1061 to 1064, an 
end signal indicating the end of the processes in the de- 
tecting process parts 1041 A to 1044C etc. 
[0035] The integrating process part control signals 
1 121 A to 121 4C are used for communication between 
the integrating process parts 1051 A to 1054C and the 
control part 1 000, and are composed of a process part 
select signal for selecting the process part to be oper- 
ated among the integrating process parts 1051 A to 
1 054C, an output signal for permitting the output of the 
process results to the global data line 1030, an end sig- 
nal indicating the end of the processes in the integrating 
process parts 1051 A to 1054C etc. 
[0036] External l/Ps 1 1 30, 1 1 61 , 1 1 62, 1 1 63 and 1 1 64 



are respectively connected with the global data line 
1030 and local data lines 1061, 1062, 1063 and 1064, 
and the process results of the integrating process parts 
1051 A to 1054C and of the detecting process parts 
1041 A to 1044C in the course of the operation thereof 
or the Interim process results retained In the Integrating 
memories 1011 to 1014 or in the detection memories 
1071 to 1074 can be taken out to the exterior through 
such external l/Fs. 

[0037] In the following, there will be explained the 
function of the configuration shown In Figs: 1 A and 1 B, 
in case of forming a neural network for image recogni- 
tion by parallel hierarchical process. At first there will be 
explained In detail the content of process of the neural 
network with reference to Fig. 2. This neural network 
processes, In hierarchical manner, the Information relat- 
ing to the recognition (detection) of an object or geomet- 
rical feature, and has a basic structure of so-called con- 
volutions! network structure (LeCun, Y. and Bengio, Y. t 
1 995, "Convolution al Networks for Images Speech, and 
Time Series'* In Handbook of Brain Theory and Neutral 
Networks (M. Arbib, Ed.), MIT Press, pp.255-258). The 
output from the final layer (uppermost layer) is a cate- 
gory of the recognized object as the result of recognition 
and positional information thereof on the input data. 
[0038] A data input layer 1 01 receives local area data 
from a photoelectric converting element such as a 
CMOS sensor or a CCD element. A first feature detect- 
ing layer 102 (1,0) detects local low-level features (ge- 
ometrical features such as specified directional compo- 
nents and specified spatial frequency components and 
color component features) of the image pattern entered 
from the data input layer 1 01 , in local areas around var- 
ious center positions in the entire image area (or local 
areas around predetermined sampling points over the 
entire image area), by a number of plural feature cate- 
gories in plural scale levels or plural resolutions in each 
same point. 

[0039] A feature integrating layer 1 03 (2,0) has a pre- 
determined receiving field structure (hereinafter receiv- 
ing field means the coupling range with the output ele- 
ment of an Immediately preceding layer, and receiving 
field structure means the distribution of the coupling 
weights), and executes integration (calculation such as 
local averaging or sub-sampling by detection of maxi- 
mum output) of the outputs of the plural neuron ele- 
ments present within a same receiving field. Such inte- 
grating process has a function of spatially blurring the 
output from the feature detecting layer 1 02 (1 ,0) thereby 
tolerating positional aberration or deformation. The re- 
ceiving fields of the neurons in the feature integrating 
layer have a common structure among the neurons with- 
in a same layer. 

[0040] Like the aforementioned layers, succeeding 
feature detecting layers 102 ((1 ,1), (1 ,2),..., (1,M)) exe- 
cute detection of plural different features and feature in- 
tegrating layers 103 ((2,1), (2,2),..., (2,M)) execute inte- 
gration of the detection results on the plural features 
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from the preceding feature detecting layers. However, 
the former feature detecting layers are so coupled 
(wired) as to receive the outputs of the cell elements of 
the feature integrating layer of the preceding stage be- 
longing to the same channel. The sub-sampling to be 
executed in the feature Integrating layer executes, for 
example, averaging of the outputs from the local areas 
(local receiving fields of the neurons of such feature in- 
tegrating layer) from the feature detecting ceil assembly 
of the same feature category. 
[0041] In the following, as a specific example, there 
will be explainedthe process of detecting an eye pattern 
from an input image, with reference to Figs. 1 to 5. 
[0042] Fig. 3 is a flow chart for detecting an eye pat- 
tern from the input image. In a step S301, an image is 
entered into the auxiliary memory 1015, which corre- 
sponds to the data input layer 1 01 . Then, in a step S302, 
there is detected a primary feature. The primary features 
in the eye detection are for example as shown in Fig. 4. 
[0043] More specifically, the features of specified di- 
rections such as vertical (4-1-1), horizontal (4-1-2), di- 
agonal upward to the right (4-1-3) and diagonal down- 
ward to the right (4-1-4) are extracted. Also secondary 
features are a V-pattem open to the right (4-2-1), a V- 
pattem open to the left (4-2-2) and a circle (4-2-3), and 
tertiary feature is an eye pattern (4-3-1). The detecting 
process parts 1 041 A to 1 044C are so constructed as to 
respectively detect these features, and the detecting 
process parts 1 041 A, 1 042A, 1 043A and 1 044A detect 
the primary features, respectively the vertical (4-1-1), 
horizontal (4-1-2), diagonal upward to the right (4-1-3) 
and diagonal downward to the right (4-1-4). Similarly the 
detecting process parts 1 041 B and 1 042B detect the 
secondary features, respectively the V-pattem open to 
the right (4-2-1) and the V-pattern open to the left 
(4-2-2), and the detecting process part 1041C detects 
the eye (4-3-1). 

[0044] Since the present example of eye detection 
employs 4 primary features, 3 secondary features and 
i tertiary feature, there are only used the detecting proc- 
ess parts 1 041 A to 1 044A, 1 041 B to 1 043B and 1 041 C, 
while those 1044B and 1042C to 1044C are not used. 
[0045] The primary feature detection in the step S302 
corresponds to the feature detecting layer 1 02 ( 1 ,0), and 
each detecting process part corresponds to the detect- 
ing module 1 04 forthe feature f . The integrating memory 
controller 1010 controls the auxiliary memory control 
signal 1025 to read local data around a certain center 
point in the image (such local area corresponds to the 
receiving field 105), and outputs such data to the global 
data line 1 030. These data are entered into the detecting 
process parts 1 041 A to 1 044A at a same timing in par- 
allel manner for respective detections of the aforemen- 
tioned primary features. 

[0046] In this state, in the detecting process part con- 
trol signals 1111Ato111 4A, the process part select sig- 
nals alone are in the enabled state. Then, in response 
to the end signal indicating the end of the process, the 



control part 1 000 shifts the output signals to the enabled 
state, whereby the detecting process parts 1041 A to 
1 044A output the process results to the local data lines 
1061 to 1064. At the same time the detection memory 
5 controller 1 090 outputs the address to the detection ad- 
dress line 1 080 and controls the detection memory con- 
trol signals 1081 to 1084 thereby retaining the data on 
the local data lines in the detection memories 1071 to 
1074. 

10 [0047] The process in each detecting process part 
employs the Input data and the coupling weight. For ex- 
ample, in detecting the vertical (4-1-1) of the primary 
feature, there Is executed a multiplication-summation 
calculation between the receiving field of a size of 3*3 

15 and the receiving field structure conceptually shown in 
Fig. 5 with the coupling weight of 0 or 1 (hereinafter the 
receiving field means the coupling range with the output 
element of the Immediately preceding layer and the re- 
ceiving field structure means the distribution of the cou- 

20 pling weights). In this step S302, the primary feature de- 
tection is executed over the entire image area by chang- 
ing the center point of the local area read from the aux- 
iliary memory 1015 to each point in the entire image or 
to each of the predetermined sampling points in the en- 

25 tire image. Such processing over the entire image by 
moving the local area is executed similarly in the suc- 
ceeding integrating process or in the secondary or ter- 
tiary feature detection. 

[0048] A succeeding step S303 integrates the primary 

30 features. This step corresponds to the feature integrat- 
ing layer 103 (2,0), and executes integration of the data 
retained in the detection memories 1071 to 1074 (cor- 
responding to the integration (operations such as local 
averaging or sub-sampling for example by the maximum 

35 output detection) of the outputs of the plural neuron el- 
ements present within a same receiving field in the fea- 
ture detecting layer 102 (1,0)). Also each integrating 
process part corresponds to the integrating module 1 06 
of feature f. The detection memory controller 1090 out- 

40 puts an address to the detection address line 1 080, and! 
controls the detection memory control signals 1081 to 
1 084 to read the local data of the detection memories 
1071 to 1074, whereby the local data are entered into 
the integrating process parts 1051 A to 1054A through 

45 the local data lines 1061 to 1064. 

[0049] In the detection process in the step S302, the 
detecting process parts 1041 A to 1044A receive the 
same data, but, in the integrating process in the step 
S303, the integrating process parts 1051 A to 1054A re- 

50 ceive respectively different data. However, since the po- 
sition and size in the input image of the receiving field 
in this integrating process are common to all the inte- 
grating process parts 1 051 A to 1 054A, a same address 
from the detection memory controller 1 090 can be used 

55 to indicate the data position in the detection memories 
1 071 to 1 074 and the receiving fields are therefore com- 
mon. 

[0050] Stated differently, in readingthe local data from 
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the detection memories 1071 to 1074, there are not re- 
quired different addresses for such detection memories 
but the data readouts from the detection memories can 
be executed in parallel manner by a single address out- 
put, and the integrating processes in the integrating 
process parts 1 051 A to 1054A are also executed In par- 
allel manner/The integrating process parts execute av- 
eraging of the input data, detection of the maximum val- 
ue etc. as explained in the foregoing. 
[0051) In this state, among the integrating process 
part control signals 11 21 A to 1124A, the process part 
select signals alone are in the enabled state. Then, in 
response to the end signal indicating the end of the proc- 
ess, the control part 1 000 shifts the output signals to the 
enabled state in succession, whereby the integrating 
process parts 1051 A to 1054A output the process re- 
sults to the global data line 1030 on time-shared basis. 
At the same time the integrating memory controller 1010 
outputs an address to the integrating address line 1020 
and controls the integrating memory control signals 
1021 to 1024 thereby retaining the data on the global 
data line in succession in the integrating memories 1011 
to 1014. By matching the timing of enabling the output 
signals of the Integrating process part control signals 
1 1 21 A to 1 1 24A with that of enabling the memory select 
signals of the integrating memory control signals, it is 
rendered possible to retain the outputs of the integrating 
process parts 1 051 A, 1 052A, 1 053A and 1 054A respec- 
tively in the integrating memories 1011, 1012, 1013 and 
1014. 

[0052] Through the above-described steps, the inte- 
grating memory 1011 retains the result of integrating the 
detection results of the vertical directional primary fea- 
ture, the integrating memory 1012 retains the result of 
integrating the detection results of the horizontal direc- 
tion, the integrating memory 1013 retains the result of 
integrating the detection results of the diagonal direction 
upward to the right, and the integrating memory 1014 
retains the result of integrating the detection results of 
the diagonal direction downward to the right. 
[0053] A step S304 executes detection of the second- 
ary features, corresponding to the feature detecting lay- 
er 102 (1,1). The secondary features consist of V- 
shapes (4-2-1 , 4-2-2) and circle (4-2-3) as shown In Fig. 
4. The Vrshape can be detected by the detection of two 
diagonals (4-1-3, 4-1-4) in the primary features and the 
positional relationship thereof, and the circle can be de- 
tected by the detection of all the primary features in the 
receiving field and the positional relationship thereof. 
Stated differently, the secondary feature can be detect- 
ed by combining the primary features of plural kinds. 
[0054] The detection processes of these secondary 
features are executed in the detecting process parts 
1041B to 1043B. The integrating memory controller 
1 01 0 outputs an address to the integrating address line 
1020 and controls the integrating memory control sig- 
nals 1021 to 1024 to read the integrated local data of 
the primary features, retained in the integrating memo- 



ries 1 01 1 to 1 014, for output to the global data line 1 030. 
In this operation, the memory select signals in the inte- 
grating memory control signals 1 021 to 1 024 are shifted 
to the enabled state in succession, whereby the inte- 

s grated primary features are outputted in succession in 
the order of the Integrating memories 1 011 to 1014. Stat- 
ed differently, the global data line 1030 is used on time- 
shared basis. As in the step S302, these data are en- 
tered into the detecting process parts 1041B to 1043B 

10 at a same timing in parallel manner, for the purpose of 
the detection of the aforementioned secondary features. 
[0055] In this operation, since the secondary features 
to be detected are 3 kinds, in the detecting process part 
control signals, the process part select signals 1111 B to 

15 1113B alone are in the enabled state. Then, in response 
to the end signal indicating the end of process, the con- 
trol unit 1000 shifts the output signals to the enabled 
state, whereby the detecting process parts 1041B to 
1 043B output the process results to the local data lines 

20 1 061 to 1063. At the same time the detection memory 
controller 1090 outputs an address to the detection ad- 
dress line 1080 and controls the detection memory con- 
trol signals 1081 to 1083 to retain the data on the local 
data lines in the detection memories 1071 to 1073. 

25 [0056] A succeeding step S305 integrates the sec- 
ondary features. This step corresponds to the feature 
integrating layer 103 (2,1), and executes integration of 
the data retained in the detection memories 1071 to 
1 073. The detection memory controller 1 090 outputs an 

30 address to the detection address line 1 080, and controls 
the detection memory control signals 1081 to 1083 to 
read the local data of the detection memories 1071 to 
1 073, whereby the local data are entered into the inte- 
grating process parts 1 051 B to 1 053B through the local 

35 data lines 1061 to 1063. The integrating process parts 
execute averaging of the input data, detection of the 
maximum value etc. as in the step S303. In this state, 
among the integrating process part control signals, the 
process part select signals 1 1 21 B to 1 1 23B alone are in 

40 the enabled state. Then, in response to the end signal 
indicating the end of the process, the control part 1000 
shifts the output signals to the enabled state in succes- 
sion, whereby the integrating process parts 1051B to 
1 053B output the process results to the global data line 

45 1 030 on time-shared basis. At the same time the inte- 
grating memory controller 1010 outputs an address to 
the integrating address line 1020 and controls the Inte- 
grating memory control signals 1 021 to 1 023 thereby re- 
taining the data on the global data line in the integrating 

50 memories 1011 to 1013. 

[0057] A step S306 executes detection of the tertiary 
feature, corresponding to the feature detecting layer 1 02 
(1,2). The tertiary feature consists of an eye pattern 
(4-3-1) as shown in Fig. 4. The eye pattern can be de- 

55 tected by the detection of all the secondary features (V- 
shapes (4-2-1 , 4-2-2) and circle (4-2-3)) in the receiving 
field and the positional relationship thereof. Stated dif- 
ferently, the tertiary feature can be detected by combin- 
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ing the secondary features of plural kinds. The detection 
process of the tertiary feature Is executed In the detect- 
ing process part 1 041 C. 

The integrating memory controller 1010 outputs an ad- 
dress to the integrating address line 1020 and controls 
the Integrating memory control signals 1021 to 1023 to 
read the integrated local data of the secondary features, 
retained in the integrating memories 1011 to 1013, for 
output to the global data line 1030. In this operation, as 
in the step 304, the memory select signals in the inte- 
grating memory control signals 1 021 to 1 024 are shifted 
to the enabled state in succession, whereby the inte- 
grated primary features are outputted in succession in 
the order of the integrating memories 1011 to 1013, 
whereby the global data line 1030 is used on time- 
shared basis. These data are entered Into the detecting 
process part 1041C, for the purpose of the detection of 
the aforementioned tertiary feature. In this operation, 
since the tertiary feature to be detected is only 1 kind, 
the process part select signal 1 1 11 C alone in the detect- 
ing process part control signals Is In the enabled state. 
Then, in response to the end signal indicating the end 
of process, the control unit 1 000 shifts the output signals 
to the enabled state, whereby the detecting process part 
1041 C outputs the process result to the local data line 
1 061 . At the same time the detection memory controller 
1090 outputs an address to the detection address line 

1080 and controls the detection memory control signal 

1081 to retain the data on the local data line In the de- 
tection memory 1 071 . 

[0058] A succeeding step S307 integrates the tertiary 
feature. This step corresponds to the feature integrating 
layer 103 (2,2), and executes integration of the data re- 
tained in the detection memory 1071. The detection 
memory controller 1 090 outputs an address to the de- 
tection address line 1080, and controls the detection 
memory control signal 1081 to read the local data of the 
detection memory 1071 , whereby the local data of the 
tertiary feature are entered into the integrating process 
part 1051 C through the local data line 1061. The inte- 
grating process part executes averaging of the input da- 
ta, detection of the maximum value etc. Then, in re- 
sponse to the end signal indicating the end of the proc- 
ess, the control part 1 000 shifts the output signals to the 
enabled state in succession, whereby the integrating 
process part 1051C outputs the process result to the 
global data line 1030. At the same time the integrating 
memory controller 1010 outputs an address to the inte- 
grating address line 1020 and controls the integrating 
memory control signal 1021 thereby retaining the data 
on the global data line in the integrating memory 1 011 . 
[0059] The result retained in the integrating memory 
1011 constitutes the final result of the eye detection. It 
is also possible to dispense with the aforementioned 
step S307 and to utilize the result retained in the detec- 
tion memory 1071 as the result of eye detection. 
[0060] As explained in the foregoing embodiment, the 
present invention allows to execute the detection proc- 



ess of plural features and the integration of the results 
of such detection easily in parallel manner, and it Is also 
easy to execute such process in hierarchic manner, rt Is 
furthermore possible to prevent complication in the wir- 
s ings, even in the processing of a complex receiving field 
structure, by temporarily retaining the results of detec- 
tion or integration in memories and then entering such 
resu Its into the processors through the data lines by suit- 
ably designating the addresses of such memories. It Is 
io furthermore possible to read out the results of each 
process or to read the result of an arbitrary position, re- 
tained In the memory, by addressing thereto. 
[0061] The processes in the detecting process parts 
or in the integrating process parts in the present embod- 
15 iment may be achieves by digital processes employing 
a DSP or the like, or by analog processes of converting 
current value or pulse width in an analog circuit. In case 
of digital process, the memory can be composed of a 
digital memory, and the data line can be constructed as 
20 a bus. In case of analog process, the memory can be 
composed of an analog memory In which an amount Is 
retained for example by a charge amount, and the proc- 
ess can be executed by representing an amount by a 
pulse width and conducting a processing by a pulse 
width modulation. In such configuration, there may be 
added a circuit for converting such charge amount into 
a pulse width and a circuit for inverse conversion, to the 
data input/output lines of the memory. Regardless of the 
digital or analog processing, the configuration of the 
present invention easily enables hierarchic pattern rec- 
ognition. 

[0062] In the following there will be explained a case 
of adopting the pattern detection (recognition) appara- 
tus of the foregoing embodiment in an image pickup ap- 
paratus, thereby achieving focusing to a specified object 
or color correction or exposure control for the specified 
object with reference to Fig. 12, which shows the con- 
figuration of an image pickup apparatus employing the 
pattern detection (recognition) apparatus of the forego- 
ing embodiment 

[0063] Referring to Fig. 1 2, an image pickup appara- 
tus 5101 is provided with an imaging optical system 
5102 including an image taking lens and a zooming 
drive control mechanism, a CCD orCMOS sensor5103, 
a measuring unit 51 04 for image pickup parameters, an 
image signal processing circuit 5105, a memory unit 
51 06, a control signal generation unit 51 07 for generat- 
ing control signals for controlling the image pickup op- 
eration and the image pickup conditions, a display 51 08 
serving also as a view finder such as an EVF (electronic 
view finder), a strobe flash unit 51 09 and a memory me- 
dium 5110, and is further provided with the aforemen- 
tioned pattern detection apparatus as an object detec- 
tion (recognition) apparatus. 

[0064] The image pickup apparatus 5101 executes, 
by the object detection (recognition) apparatus 5111, the 
detection (detection of position and size) of the face im- 
age of a preregistered person in the taken image. When 
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the information on the position and size of such person 
is entered from the object detection (recognition) appa- 
ratus 5111 to the control signal generation unit 5107, 
such control signal generation unit 5107 generates a 
control signal for optimizing the focus control, exposure 
condition control, white balance control etc. for such per- 
son, based on the output from the image pickup param- 
eter measuring unit 5104. 

[0065] Thus, the aforementioned pattern detection 
(recognition) apparatus can be applied to the image 
pickup apparatus to achieve detection of the person and 
optimum control of the image taking operation based on 
such detection. 

(Second embodiment) 

[0066J Figs. 6A and 6B show the configuration of the 
present embodiment, wherein components equivalent 
to those in Figs. 1 A and 1 B are represented by like num- 
bers. In comparison with the first embodiment, the new 
components in Figs. 6A and 6B are a control part 2000, 
an integrating memory controller 2010, sub-global data 
lines 2031 to 2034, global data line switches 2131 to 
2133, and switch control lines 2231 to 2233. 
[0067] In contrast to the first embodiment in which the 
global data bus 1 030 shown in Figs. 1 A and 1 B are con- 
nected to ail the integrating memories 1011 to 1014, the 
auxiliary memory 1015, all the detecting process parts 
1041 A to 1044C and all the integrating process parts 
1051 A to 1054C, a sub-global data line in the present 
embodiment is connected to an integrating memory, plu- 
ral detecting process parts and plural integrating proc- 
ess parts or auxiliary memories. The number of the de- 
tecting process parts and the integrating process parts 
connected to a sub-global data line is basically equal to 
the number of layers in the hierarchic process. For ex- 
ample, in the present embodiment, there is assumed a 
hierarchic process of three layers as in the first embod- 
iment, so that the sub-global data line 2031 is connected 
to an integrating memory 1011 , detecting process parts 
1 041 A to 1 041C and integrating process parts 1051 A to 
1051C. 

[0068] Also each of the global data line switches 2 131 
to 21 33 is connected to the adjacent two of the sub-glo- 
bal data lines 2031 to 2034. The switch control lines 
2231 to 2233 are connected to the global data line 
switches 2131 to 2133 and the control part 2000. The 
global data lines switches 2131 to 2133 respectively 
connect or disconnect the sub-global data lines 2031 to 
2034 based on the switch control signals 2231 to 2233. 
[0069] In the following there will be explained the func- 
tions specific to the present embodiment, with reference 
to Figs. 6A, 6B, 7A and 7B. Other functions are similar 
to those explained in the embodiment 1. 
[0070] In case of entering the data retained in the aux- 
iliary memory 1 01 5 or the integrating memories 1 01 1 to 
1044 into the detecting process parts 1041 A to 1044C, 
the global data line switches 2131 to 2133 are turned 



on as shown In Fig. 7A according to the switch control 
signals 2231 to 2233 from the control part 2000, where* 
by all the sub-global data lines 2031 to 2034 are con- 
nected. In this state, therefore, the configuration be- 

5 comes substantially equal to that explained in the first 
embodiment, whereby the data outputted from the aux- 
iliary memory 1015 and the integrating memories 1011 
to 1 01 4 are respectively entered into the detecting proc- 
ess parts 1041 A to 1044C at a same timing in parallel 

10 manner. 

[0071] Also In case of retaining the results processed 
in the integrating process parts 1051 A to 1054C In the 
integrating memories 1011 to 1014, the global data line 
switches 2131 to 2133 are turned off as shown in Fig. 

15 7B according to the switch control signals 2231 to 2233 
from the control part 2000, whereby the sub-global data 
lines 2031 to 2034 are disconnected. In such state, 
therefore, the integrating process parts 1 051 A to 1 054A 
need not retain the process results in the integrating 

20 memories 1011 to 1014 on time-shared basis as ex- 
plained in the first embodiment and the integrating mem- 
ory controller 2010 need not execute data writing into 
the integrating memories 1011 to 1014 on time-shared 
basis, but the process results of the integrating process 

25 parts can be retained in the integrating memories at a 
same timing in parallel manner. 
[0072] As explained in the foregoing, the present em- 
bodiment allows to enter the data of the auxiliary mem- 
ory or the integrating memories into the detecting proc- 

30 ess parts at a same timing in parallel manner and to re- 
tain the process results of the integrating process parts 
in the Integrating memories at a same timing in parallel 
manner, thereby being capable of a process similar to 
that in the first embodiment and of reducing the process 

35 time in comparison with the first embodiment. 

(Third embodiment) 

[0073] Figs. 8A and 8B show the configuration of the 
40 present embodiment, wherein components equivalent 
to those in Figs. 1A and 1 Bare represented by like num- 
bers. In comparison with the first embodiment, the new 
components in Figs. BA and 8B are a control part 3000, 
an integrating memory controller 301 0, global data lines 
45 3031 to 3035, and detecting process parts 3041 A to 
3044C. 

[0074] In contrast to the first embodiment in which the 
global data bus 1 030 shown in Figs. 1 A and 1 B are con- 
nected to all the integrating memories 1011 to 1014,the 

so auxiliary memory 1 01 5, all the detecting process parts 
1041 A to 1044C and all the integrating process parts 
1041 A to 1044C and all the integrating process parts 
1051 A to 1054C, a global data line in the present em- 
bodiment is connected to an integrating memory or the 

55 auxiliary memory, all the detecting process parts and 
plural integrating process parts. The number of the in- 
tegrating process parts connected to a global data line 
is basically equal to the number of layers in the hierar- 



10 



17 



EP 1 262 908 A1 



18 



chic process. For example, In the present embodiment, 
there Is assumed a hierarchic process of three layers as 
In the first embodiment, so that the global data line 3031 
is connected to an integrating memory 1011, detecting 
process parts 3041 A to 3044C and integrating process 
parts 1051 A to 1051C. In Figs. 8A and 8B, with respect 
to the connection between the global data lines 3031 to 
3035 and the detecting process parts 3041 A to 3044C, 
there are only illustrated the detecting process parts 
3041 A to 3044A and other parts are omitted, 
[0075J Also each of the detecting process parts can 
receive input from all the global data lines 3031 to 3035. 
Such configuration enables to read the data from the 
integrating memories 1011 to 1014 and the auxiliary 
memory 1015 at a same timing after parallel operation 
thereof, to enter the data into all the predetermined de- 
tecting process parts 3041 A to 3044C at a same timing 
in parallel manner, and also to retain the process results 
from the integrating process parts in the integrating 
memories 1011 to 101 4 at a same timing in parallel man- 
ner. 

[0076J In the following there will be explainedthefunc- 
tions specific to the present embodiment. Other func- 
tions are similar to those explained in the embodiment 1 . 
[0077] In case of entering the data retained in the aux- 
iliary memory 1 01 5 or the process results retained in the 
integrating memories 1011 to 1044 into the detecting 
process parts 3041 A to 3044C, these parts are operated 
in parallel manner according to the integrating memory 
control signals 1021 to 1024 or the auxiliary memory 
control signal 1 025 from the integrating memory control- 
ler 301 0 and, at a same timing, the integrating memories 
1011 to 1014 output the data to the global data lines 
3031 to 3034 while the auxiliary memory 1015 outputs 
the data to the global data line 3035. In this operation, 
the integrating memories 1011, 1012, 1013 and 1014 
respectively output the data to the global data lines 
3031 , 3032, 3033 and 3034. Since the detecting proc- 
ess parts 3041 A to 3034C can receive inputs from all 
the global data lines 3031 to 3035, the data are entered 
to the detecting process parts at a same timing in par- 
allel manner. 

[0078] Also in case of retaining the results processed 
in the integrating process parts 1051 to 1054 into the 
integrating memories 1 011 to 1014, the data can be re- 
tained in the integrating memories 1011 to 1014 at the 
same time in parallel manner, since the integrating proc- 
ess parts 1051 A to 1051C are so constructed as to ex- 
ecute output to the global data line 3031 while the inte- 
grating process parts 1 052A to 1 052C are so construct- 
ed as to execute output to the global data line 3032. 
[0079] As explained in the foregoing, the present em- 
bodiment allows to enter the data of the auxiliary mem- 
ory or the integrating memories into the detecting proc- 
ess parts at a same timing in parallel manner and to re- 
tain the process results of the integrating process parts 
in the integrating memories at a same timing in parallel 
manner, thereby being capable of a process similar to 



that in the first embodiment and of reducing the process 
time in comparison with the first and second embodi- 
ments. 

5 (Fourth embodiment) 

[0080] Figs. 9A and 9B show the configuration of the 
present embodiment, wherein components equivalent 
to those in Figs. 1 A and 1 B are represented by like num- 

10 bers. In comparison with the first embodiment, the new 
components In Figs. 9A and 9B are a control part 4000, 
variable detecting process parts 4041 to 4044, variable 
integrating process parts 4051 to 4054, a circuit config- 
uration memory part 41 1 0, a circuit configuration control 

is part 4120, variable detecting process part control sig- 
nals 41 11 to 4114, and variable integrating process part 
control signals 41 21 to 4124. With respect to the varia- 
ble detecting process part control signals and the vari- 
able integrating process part control signals, Figs. 9A 

20 and 9B show the variable detecting process part control 
signal 4111 alone and omits others. 
[0081 ] In contrast to the first embodiment in which all 
the process parts necessary for the recognition process 
(namely detecting process parts and integrating proc- 

25 ess parts) are prepared in advance and the process 
parts to be used in each processing are selected from 
such prepared process parts by the control signals from 
the control part, the present embodiment is composed 
of the variable detecting process parts and the variable 

30 integrating process parts of a number capable of parallel 
operation at a same timing. 

[0082] The variable detecting process parts 4041 to 
4044 and the variable integrating process parts 4051 to 
4054 are formed by a logic IC which is called FPGA (field 

35 programmable gate array) and composed of plural cir- 
cuit blocks and wiring blocks allowing the user to con- 
struct an arbitrary logic, or by an analog IC which is 
called FPAA (E. Lee, P. Gulak, 1991, 'A CMOS Held- 
programmable Analog Array", IEEE JOURNAL OF 

40 SOLID-STATE CIRCUITS, Vol. 26, No. 12, pp. 
1 860-1 867) and which allows the user to execute an ar- 
bitrary analog process by changing the wiring between 
the analog block circuits or changing the charge amount 
in a capacitor constituting a multiplication coefficient. 

45 The circuit configuration information memory part 4110 
stores information of the circuit configuration for con- 
structing the process parts into a configuration capable 
of the arbitrary process. Also the circuit configuration 
control part 4120 serves to change the circuit configu- 

50 ration by controlling the process parts according to such 
circuit configuration information. More specifically, the 
variable detecting process parts 4041 to 4044 and the 
variable integrating process parts 4051 to 4054 are con- 
trolled according to the circuit configuration control sig- 

55 nals from the circuit configuration control part 4120, and 
are reconstructed into the circuit configuration matching 
the circuit configuration information signals from the cir- 
cuit configuration information memory part 4110. 
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[0083] In the following, the function of the present em- 
bodiment will be explained with reference to Figs. 9A, 
9B, 10 and 11, in which Fig. 10 is a flow chart showing 
the function of the present embodiment. In the following 
there will be explained a case of detecting an eye pat- s 
tern, as in the first embodiment. 
[0084] Referring to Fig. 1 0, a step S901 enters an im- 
age Into the auxiliary 1015. Then, in a step S902, in re- 
sponse to a signal from the control part 4000, the circuit 
configuration information memory part 4110 outputs a 10 
circuit configuration Information signal while the circuit 
configuration control part 4120 outputs a circuit config- 
uration control signal, and the circuits of the variable de- 
tecting process parts 4041 to 4044 are constructed ac- 
cording to such signals. In this operation, there are con- 1* 
structed circuits for detecting the primary features (cf. 
Fig. 4) in the eye pattern detection. 
[0085] In a step S903, the primary features are de- 
tected by the variable detecting process parts 4041 to 
4044 so constructed as to detect the primary features 20 
of the eye pattern. The Integrating memory controller 
1010 controls the auxiliary memory control signal 1 025 
to read the local data of the Image from the auxiliary 
memory 1015 for output to the global data line 1030. 
Such data are entered into the variable detecting proc- 25 
ess parts 4041 to 4044 at a same timing in parallel man- 
ner for the detection of primary features. Then, in re- 
sponse to the end signal indicating the end of the proc- 
ess, the control part 4000 shifts the output signals to the 
enabled state, whereby the variable detecting process 30 
parts 4041 to 4044 output the process results to the local 
data lines 1 061 to 1 064. At the same time, the detecting 
memory controller 1 090 outputs an address to the de- 
tecting address line 1080 ; and controls the detecting 
memory control signals 1081 to 1084 to retain the data 35 
on the local data lines in the detecting memories 1071 
to 1074. 

[0086] Then, in a step S904, in response to a signal 
from the control part 4000, the circuit configuration in- 
formation memory part 41 1 0 outputs a circuit configura- 40 
tion information signal while the circuit configuration 
control part 4120 outputs a circuit configuration control 
signal, and the circuits of the variable integrating proc- 
ess parts 4051 to 4054 are constructed according to 
such signals. In this operation, there are constructed cir- 45 
cuits for integrating the detected primary features by op- 
erations such as local averaging or sub-sampling by de- 
tection of maximum output. 

[0087] In a step S905, the primary features are inte- 
grated by the variable integrating process parts 4051 to so 
4054 so constructed as to integrate the primary fea- 
tures. The detecting memory controller 1 090 outputs an 
address on the detecting address line 1080, and con- 
trols the detecting memory control signals 1 081 to 1 084 
to read the local data of the detecting memories 1071 55 
to 1 074, whereby the local data are entered into the var- 
iable integrating process parts 4051 to 4054 through the 
local data lines 1061 to 1064 and the primary features 



are Integrated. Then, In response to the end signal Indi- 
cating the end of the process, the control part 4000 shifts 
the output signals to the enabled state, whereby the var- 
iable integrating process parts 4051 to 4054 output the 
process results to the global data line 1030 on time- 
shared basis. At the same time, the Integrating memory 
controller 1 01 0 outputs an address to the integrating ad- 
dress line 1020, and controls the integrating memory 
control signals 1021 to 1024 to retain the data on the 
global data line in the integrating memories 1011 to 
1014. 

[0088] In a step S906, in response to the circuit con- 
figuration Information signal and the circuit configuration 
control signal, the circuits of the variable detecting proc- 
ess parts 4041 to 4044 are reconstructed as in the step 
S902. In this operation, there are constructed circuits for 
detecting the secondary features (cf. Fig. 4) in the eye 
pattern detection (Since the secondary features are 
three kinds, there are only reconstructed the variable 
detecting process parts 4041 to 4043). 
[0089] In a step S907, the secondary features are de- 
tected by the variable detecting process parts 4041 to 
4044 so constructed as to detect the secondary features 
of the eye pattern. The integrating memory controller 
1 01 0 outputs an address to the integrating address line 
1020 and controls the integrating memory control sig- 
nals 1021 to 1024 to readthelocal data of the integrated 
primary features from the integrating memories 1011 to 
1 01 4, for output to the global data line 1 030. Such data 
are entered into the variable detecting process parts 
4041 to 4044 at a same timing in parallel manner for the 
detection of aforementioned secondary features. In this 
operation, since the secondary features to be detected 
are of three kinds, the process part select signals in the 
variable detecting process part control signals 4111 to 
4113 alone are in the enabled state. Then, in response 
to the end signal indicating the end of the process, the 
control part 4000 shifts the output signals to the enabled 
state, whereby the variable detecting process parts 
4041 to 4044 output the process results to the local data 
lines 1061 to 1063. At the same time, the detecting 
memory controller 1090 outputs an address to the de- 
tecting address line 1080, and controls the detecting 
memory control signals 1 081 to 1083 to retain the data 
on the local data lines in the detecting memories 1071 
to 1073. 

[0090] Then, in a step S908, based on the circuit con- 
figuration information and the circuit configuration con- 
trol signal, the circuits of the variable integrating process 
parts 4051 to 4053 are re-constructed. In this operation, 
there are constructed circuits for integrating the detect- 
ed primary features by operations such as local averag- 
ing or sub-sampling by detection of maximum output. 
This step S908 can naturally be dispensed with if the 
circuit configuration obtained in the step S904 can be 
used without change. 

[0091 ] In a step S909, the secondary features are in- 
tegrated by the reconstructed variable integrating proc- 
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ess parts 4051 to 4053. The detecting memory control- 
ler 1090 outputs an address on the detecting address 
line 1080, and controls the detecting memory control 
signals 1081 to 1083 to read the local data of the de- 
tecting memories 1071 to 1073, whereby the local data 
are entered into the variable Integrating process parts 
4051 to 4053 through the local data lines 1061 to 1063. 
Each variable integrating process part executes proc- 
esses such as averaging of the input data or detection 
of the maximum value. In this operation, the process 
part select signals 41 21 to 41 23 alone are In the enabled 
state in the variable integrating process part control sig- 
nals. Then, in response to the end signal indicating the 
end of the process, the control part 4000 shifts the out- 
put signals to the enabled state in succession, whereby 
the variable Integrating process parts 4051 to 4053 out- 
put the process results to the global data line 1030 on 
time-shared basis. At the same time, the integrating 
memory controller 1010 outputs an address to the inte- 
grating address line 1020, and controls the integrating 
memory control signals 1021 to 1023 to retain the data 
on the global data line in the integrating memories 101 1 
to 1013. 

[0092] In a step S910, in response to the circuit con- 
figuration information signal and the circuit configuration 
control signal, the circuit of the variable detecting proc- 
ess part 4041 is reconstructed. In this operation, there 
is constructed a circuit for detecting the tertiary feature 
(cf . Fig. 4) in the eye pattern detection. 
[0093] In a step S911 , the tertiary feature is detected 
by the reconstructed variable detecting process part 
4041 . The integrating memory controller 1010 outputs 
an address to the integrating address line 1 020 and con- 
trols the integrating memory control signals 1021 to 
1 023 to read the local data of the integrated secondary 
features from the integrating memories 1011 to 1013, 
for output to the global data line 1030. Such data are 
entered into the variable detecting process part 4041 for 
the detection of aforementioned tertiary feature. Then, 
in response to the end signal indicating the end of the 
process, the control part 4000 shifts the output signal to 
the enabled state, whereby the variable detecting proc- 
ess part 4041 outputs the process result to the local data 
line 1 061 . At the same time, the detecting memory con- 
troller 1 090 outputs an address to the detecting address 
line 1080, and controls the detecting memory control 
signal 1081 to retain the data on the local data line in 
the detecting memory 1071. 

[0094] Then, in a step S912, based on the circuit con- 
figuration information and the circuit configuration con- 
trol signal, the circuit of the variable integrating process 
part 4051 is re-constructed. This step S91 2 can natural- 
ly be dispensed with if the circuit configuration obtained 
in the step S904 can be used without change. 
[0095] In a step S913, the tertiary features are inte- 
grated by the re-constructed variable integrating proc- 
ess part 4051. The detecting memory controller 1090 
outputs an address on the detecting address line 1080, 



and controls the detecting memory control signal 1081 
to read the local data of the detecting memory 1071, 
whereby the local data of the tertiary feature are entered 
into the variable integrating process part 4051 through 
s the local data line 1061 . Then, in response to the end 
signal indicating the end of the process, the control part 
4000 shifts the output signals to the enabled state in suc- 
cession, whereby the variable integrating process part 
4051 outputs the process result to the global data line 
10 1 030. At the same time, the integrating memory control- 
ler 1010 outputs an address to the Integrating address 
line 1020, and controls the integrating memory control 
signal 1021 to retain the data on the global data line in 
the integrating memory 1011 . 
15 [0096] The result retained in the integrating memory 
1011 constitutes the final result of the eye detection. It 
is also possible to dispense with the aforementioned 
step S313 and to utilize the result retained in the detec- 
tion memory 1071 as the result of eye detection. 
[0097] In the foregoing, the reconstruction of the var- 
iable integrating process parts Is executed after the de- 
tection process by the variable detecting process parts 
and that of the variable detecting process parts is exe- 
cuted after the detection process by the variable inte- 
grating process parts, but, since the variable detecting 
process parts and the variable integrating process parts 
are not operated at the same time, it is possible, as 
shown in Fig. 11, to simultaneously execute the detect- 
ing process by the variable detecting process parts and 
the reconstruction of the variable integrating process 
parts or the integrating process by the variable integrat- 
ing process parts and the reconstruction of the variable 
detecting process parts. In Fig. 11, the abscissa indi- 
cates the flow of operations, and "detection" means the 
detecting process and "integration 0 means the integrat- 
ing process. Also "construction/reconstruction" means 
construction or reconstruction of the circuits. 
[0098] In the present embodiment, as explained in the 
foregoing, process parts variable in the circuit configu- 
ration are employed with the reconstruction of the cir- 
cuits according to the process in each layer, so that the 
present embodiment is capable of recognition process 
similar to that in the first embodiment and allows to re- 
duce the magnitude of the circuits of the process parts 
in comparison with the first embodiment. Also such con- 
figuration can suppress the increase in the entire proc- 
ess time, since the process and reconstruction of the 
detecting process parts and the integrating process 
parts can be executed alternately 
[0099] It is also possible to combine the present em- 
bodiment with the second or third embodiment. 
[0100] The embodiments explained in the foregoing, 
by utilizing the common data line, local data lines and 
memories, provide an advantage of executing in parallel 
manner a hierarchical process of executing parallel 
process with plural process parts on plural results ob- 
tained by plural process parts, by a simple circuit con- 
figuration not requiring complex wirings. In the detecting 
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process, a parallel operation Is rendered possible by en- 
tering same time Into the plural predetermined detecting 
process parts, and It Is also rendered possible to spa- 
tially Integrate the process results of the process parts. 
Also In the Integrating process, It is rendered possible 
to execute the Integrating process In parallel manner by 
employing a common address, Indicating the position In 
the memories, for the plural detecting memories retain- 
ing the results of different feature detections. There is 
also provided an advantage that such detecting process 
and Integrating process can be repeated by any number 
of times. 

[01 01 J It is also possible to read the interim result of 
each layer from the memory, and to change the process 
of the preceding layer for example by observing such 
interim result. There can also be provided an advantage 
of preventing the increase in the process time, in case 
of employing process parts capable of circuit recon- 
struction, by alternately executing the detecting process 
and integrating process, and the reconstruction of the 
detecting process circuit and that of the integrating proc- 
ess circuit 

[0102] Although the present invention has been de- 
scribed in its preferred form with a certain degree of par- 
ticularity, the invention is not limited to the specific em- 
bodiments above. 



Claims 

1 . A pattern recognition apparatus for detecting a pre- 
determined pattern contained in an input signal, 
comprising: 

plural detecting processing means for detecting 
respectively different features for a same input; 
plural integrating processing means for spatial- 
ly integrating, for each process results, the fea- 
tures detected by said plural detecting process- 
ing means; 

plural detecting memories for retaining the 
process results of said detecting processing 
means; 

plural Integrating memories for retaining the 
process results of said integrating processing 
means; 

a common data line to which all said predeter- 
mined detecting processing means and all said 
predetermined Integrating memories are con- 
nected at a certain timing; and 
plural local data lines each of which is connect- 
ed to a predetermined set of said detecting 
processing means, said integrating processing 
means and said detecting memory; 

wherein, in entering the process results of 
said detecting processing means retained in said 
detecting memories into said integrating processing 



means, data of a same address in said plural de- 
tecting memories are read and entered into said In- 
tegrating processing means; and 

in entering the process results of said integrat- 
5 ing processing means retained in said integrating 
memories Into said detecting processing means, 
data of a same address in said plural detecting 
memories are read arid entered into said detecting 
processing means. 

10 

2. A pattern recognition apparatus according to claim 
1, wherein, in entering the process results of said 
integrating processing means into said integrating 
memories and/or in entering the data of said inte- 
rs grating memories into said detecting processing 
means, said integrating processing means and said 
integrating memories are so controlled as to use 
said common data line on time-shared basis. 

20 3. A pattern recognition apparatus according to claim 
1, wherein said common data line includes plural 
sub common data lines and plural switches; 

each of said sub common data lines is connect- 
25 ed one of said integrating memories; 

in entering the process results of said plural in- 
tegrating processing means into sajd integrat- 
ing memories, said plural sub common data 
lines are utilized; and 
30 in entering the process results of said integrat- 

ing processing means, retained in said integrat- 
ing memories, into said detecting processing 
means, said plural sub common data lines are 
connected by controlling said switches and 
35 used as an imaginary single data line. 

4. A pattern recognition apparatus according to claim 
1 , wherein said common data line includes plural 
shared common data lines; 

40 

each of said shared common data lines is con- 
nected to one of said predetermined integrating 
memories and all said predetermined detecting 
processing means; and 
45 in entering the process results of said plural in- 

tegrating processing means into said plural in- 
tegrating memories and/or In entering the proc- 
ess results of said integrating processing 
means, retained in said Integrating memories, 
50 into said detecting processing means, said plu- 

ral shared data lines are used. 

5. A pattern recognition apparatus according to claim 
1, further comprising: 

55 

configuration information memory means for 

storing configuration information; and 

circuit configuration control means for recon- 
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strutting the circuit configuration of said detect- 
ing processing means and said integrating 
processing means, based on the configuration 
information of said configuration information 
memory means. 5 

6. A pattern recognition apparatus according to claim 
5, wherein said circuit configuration control means 
executes such control as not to simultaneously ex- 
ecute the reconstruction of the circuit configuration io 
of said detecting processing means and the recon- 
struction of the circuit configuration of said integral- 

ing processing means. 

7. A pattern recognition apparatus according to claim 15 
1 , further comprising an external interface connect- 
ed to said common data line, wherein the results in 
said detecting memories and in said integrating 
memories are read out through said external inter- 
face as an interim result of the process. 20 



25 



30 



35 



40 



45 



50 



55 



15 



EP 1262 908 A1 



o 

CD 



CD 

u; 



< 


en 






CD 


CD 


Li- 





CD 



CD 



!8 



CD 



CD , 
i rl Sag 

ig 

CDS 



CD" 



g 



Q 



T~ LLJ UJ g 



CO 



CD 

pi 



1 



r 1 



CO 



o 

C30 



CDS 



cr 



CD 



s 



CO 
CO 



CM 



is 



CD 



Eg 



5 

OL. 

zco 
pco 

So 

Eg 

QQ. 



O 



UJO 
tec 

£~ CL_ 



1 



i= CO* 

UU DC 
OQ- 



gg 0 



si 
^5 



5 CC 



-ujo 



O 

1^7 



7 

CM 

s 



ego 



CD 

C30 



5500) 



C\J 

o 



o 

CvJ 

o 



> 



CD 

CD 

p 



8 

o 



CO 

5 



CO 
CvJ 
O 



16 



EP 1 262 908 A1 



3 



p. 



UJ 



. DETECTING - 
MEMORY ■ 


" EXTERNAL 
I/F 


_S g 

LU 


. DETECTING ■ 
MEMORY • 


EXTERNAL 
* I/F 



< 

05 



S 



< 



1 

CD 
OS 

Luce 

QCu 

7 



ii 

IS 

= Qu 

Tt 



< < 

CO CO 



OQ 



CO 

s 



i 



I 

o 

2: CO 
crco 
c3lu 

Is 

QQ- 



ii 



3 



10 

o 



7 



g8 

So- 



V 



3S 

O CD 



3 



UJ 

• LI— 



,00 



<SOW 



V 



17 



EP1 262 908 A1 




18 



EP 1262 908 A1 



FIG. 3 



( START ) 



INPUT IMAGE 



DETECT PRIMARY FEATURE 



INTEGRATE PRIMARY FEATURE 



DETECT SECONDARY FEATURE 



INTEGRATE SECONDARY FEATURE 



DETECT TERTIARY FEATURE 



INTEGRATE TERTIARY FEATURE 



S301 
S302 
S303 
S304 
S305 
S306 
S307 



( END ) 



19 



EP 1 262 908 A1 



r 



FIG. 4 



PRIMARY 
FEATURE 



(4-1-1) 



(4-1-2) 




(4-1-3) 




(4-1-4) 



SECONDARY 
FEATURE 




(4-2-1) 




(4-2-2) 




(4-2-3) 



FIG. 5 



TERTIARY 
FEATURE 



0 


1 


0 


0 


1 


0 


0 


1 


0 



20 



EP 1262 908 A1 



s 


en 




CO 


CD 


CD 


Li- 


LU 



DC 



CD 

is:.. — . 

siic' 

□ 500) 



CVJ 
CO 

cr> 



CVJ 

CD 




7 



S co S muu r: S 

CVJ Kj rvi 22/ 



CVJ 0|§' 

coo 



^CD 
coco 



LU 



p 

cd or 




CD 



I 



1^ 



CD 



i 



Cvj 



r 

coo 



CVJ 

o 



CD 




21 



EP 1 262 908 A1 




22 



EP 1 262 908 A1 



FIG. 7 A 



2131' 



2132- 



2133' 



7 

Ml 



-2031 

ON 
-2032 
ON 



-2033 
ON 
-2034 



FIG. 7B 



2131 



2132- 



2133' 



t 



t 



1 



-2031 

OFF 
-2032 

OFF 
-2033 
OFF 



-2034 



23 



EP 1 262 908 A1 




24 



EP1 262 908 A1 




CD 








IGRATIN 




JXILIAR^i 
EMORY 





















,8 



sill* 



25 



EP 1 262 908 A1 





CO 




o> 


d 


CD 


n: 






26 



EP 1262 908 A1 



LU 



8 

CD 



CD 



3> 



32^ 



3 

o 



CD 



OS 



1 



PART 


PART 


VARIABLE 

DETECTING 

PROCESSING 


VARIABLE 

INTEGRATION 

PROCESSING 



CO 

to 



DC 



l 

O 
OS 
LUZCO 

<ujO 
<lua: 



ii 



lo 



(DO 



io 

)UJ 



o 



in 

'5 



CD 

.CO 



8 CO 



27 



EP 1262 908 A1 



FIG. 10 



( START ) 



INPUT IMAGE 
I 



COMPOSE VARIABLE 
DETECTING PROCESSING PART 



DETECT PRIMARY FEATURE 

} ~ 



COMPOSE VARIABLE 
INTEGRATING PROCESSING PART 



INTEGRATE PRIMARY FEATURE 

3 — 



RECOMPOSE VARIABLE 
DETECTING PROCESSING PART 



I 



JT 



DETECT SECONDARY FEATURE 

x 



RECOMPOSE VARIABLE 
INTEGRATING PROCESSING PART 



INTEGRATE SECONDARY FEATURE 
\ ~ 



RECOMPOSE VARIABLE 
DETECTING PROCESSING PART 

i ~ 



DETECT TERTIARY FEATURE 



RECOMPOSE VARIABLE 
INTEGRATING PROCESSING PART 



INTEGRATE TERTIARY FEATURE 



S901 

S902 
S903 

S904 
S905 

S906 
S907 

S908 
S909 

S910 
S911 

-S912 
S913 



( END ) 



28 



EP 1 262 908 A1 









I 


z 

1 


j 




J 

I 


| 


j 

1 


1 


| 




C£ 
DC 


[ 


INTEGRATING 




DETECTING 


1 






< 

■ 
1 


CD 

1 


INTEGRATING 




DETECTING 


COMPOSING 






| 




1 




VARIABLE 
DETECTING 
PROCESSING 
PART 


VARIABLE 
INTEGRATING 
PROCESSING 
PART 



29 



EP 1 262 908 A1 



CD 




o 

o 
o 



rn — H- 



5/ 



cm 
o 

lo 



J 



o 
a_ 
O 
O 

s 



CO. 



30 



EP 1 262 908 A1 



J 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 
EP 02 25 3806 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document wtth Indication, where appropriate, 
of relevant passages . 



Relevant 
to claim 



CLASSIFICATION OF THE 
APPLICATION 0nLCt.7) 



NEVEU J N P ET AL: "Implementation of the 
neocognltron on a SIMD architecture" 
INTELLIGENT INFORMATION SYSTEMS, 1994. 
PROCEEDINGS OF THE 1994 SECOND AUSTRALIAN 
AND NEW ZEALAND CONFERENCE ON BRISBANE, 
OLD. , AUSTRALIA 29 N0Y.-2 DEC. 1994, NEW 
YORK, NY, USA, IEEE, 
29 November 1994 (1994-11-29), pages 
179-183, XP010136681 
ISBN: 0-7803-2404-8 

* abstract * 

* page 179, left-hand column, paragraph 2 

- right-hand column, paragraph 3 * 

* page 180, left-hand column, paragraph 3 
# 

* page 181, right-hand column, paragraph 2 
* 

* page 181, left-hand column, paragraph 2 

- paragraph 3 * 

TAKAYUKI IT0 ET AL: "EXAMINATION OF 
IMPLEMENTING A NEURAL NETWORK ON A 
PARALLEL COMPUTER - NEOC0GNITR0N ON NCUBE" 
SYSTEMS & COMPUTERS IN JAPAN, SCRIPTA 
TECHNICA JOURNALS . NEW YORK, US, 
vol. 22, no. 6, 1991, pages 1-9, 
XP000259485 
ISSN: 0882-1666 

* figure 2 * 

* page 1, right-hand column, paragraph 2 * 
right-hand column, paragraph 3 * 
left-hand column, paragraphs 



1,2,7 



* page 2, 

* page 3, 
1,4,5 * 

* page 3, 



right-hand column, paragraph 2 * 
-/— 



The present search report has been drawn up for ait claims 



1,2,7 



606K9/66 
606N3/O4 



TECHNICAL FIELDS 
SEARCHED (tnLCt.7) 



G06K 
606N 
606 F 



MUNICH 



Date ol oomptettonof tho tttvdh 

11 September 2002 



Muller, M 



CATEGORY OF CITED DOCUMENTS 

X : particularly relevant It taken alone 

V : particularly relevant H combined with another 

document of the same category 
A : technotoolcaJ background . , 
O : non-written disclosure 
P : trrtermedBte document 



T ! theory or principle underlying 
E : earter patent document, but published on, or 

after the filing date 
D: document cited tn the application 
L : document cited for other reasons 



& : member of the same patent lamlly, corresponding 
document 



31 



EP 1 262 908 A1 



J 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 02 25 3806 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document vrfth Indication, where appropriate, 
of relevant c 



Relevant 
to claim 



CLASSIFICATION OF THE 
APPLICATION (lntCL7) 



US 5 519 811 A (YONEDA HIDEKI ET AL) 

21 Maty 1996 (1996-05-21) 

* abstract; claim 1; figures 3,4A,6,12-16 



1-7 



column 1, line 31 - line 47 * 

column 3, line 7 - line 25 * 

column 5, line 47 - column 6, line 65 * 

column 10, Hne 22 - Hne 37 * 

column 11, line 13-19 * 



W0 97 29437 A (SARN0FF CORP) 
14 August 1997 (1997-08-14) 

* abstract * 

* page 5, line 1 - line 31 * 



1-7 



TECHNICAL FIELDS 
SEARCHED (tat.CI.7) 



The present search report has been drawn up for all claims 



Ptoea of March 

MUNICH 



Dbto of coaptation of (he search 

11 September 2002 



MQller, M 



CATEGORY OF CITED DOCUMENTS 

X : particularly relevant rt taken alone 

Y : particularly relevant I! combined with another 

document of file same category 
A : technological background 
O : rcn-wrttten disclosure 
P : Intermediate document 



T : theory or principle LrderMng the Invention 
E : earlier patent document but published on, cr 

after the filing date 
D : document cited In the application 
L : document dtod for other reasons 



& : member of the same patent family, corresponding 
document 



32 



EP 1262 908 A1 



ANNEX TO THE EUROPEAN SEARCH REPORT 
ON EUROPEAN PATENT APPLICATION NO. 



EP 02 25 3806 



This annex lists the patent family members relating to the patent documents ched in the above-mentioned European search report 
The members are as contained In the European Patent Office EDP file on 

The European Patent Office Is In no way Sable for these particulars wtifch are merer/ given for the purpose of information. 

11-09-2002 



Patent document 
cited in search report 



Publication 



Patent family 



Publication 



US 5519811 



21-05-1996 W0 



9308538 Al 



W0 9729437 



14-08-1997 



EP 
JP 
W0 
US 



0892956 Al 
2000506642 T 
9729437 Al 
6018728 A 



3 

s _ _ 

o 

S) For more details about this annex :see Official Journal of the European Patent Office, No. 12/82 



29-04-1993 



27-01-1999 
30-05-2000 
14-08-1997 
25-01-2000 



33 



THIS PAGE BLANK (uspto) 



